In a virtual reality environment, a user is presented with sensory input that is computer generated, often in response to user actions, such as actions that are sensed from VR/AR (virtual reality/augmented reality) input devices such as head-mounted displays (“HMDs”) or hand-held VR controllers. The outputs of a virtual reality computer system might include visual output, audio output, touch output (haptic, pressure, temperature, etc.) and other outputs. The outputs are presented to the user in ways that immerse the user. For example, the user might be wearing a headgear that covers the user's eyes to immerse the user in an experience. The headgear might provide a processor with sensor inputs about head movement. If the processor creating the visual output can modify that visual output based on the sensor inputs so that the user perceives the scenery to be stable relative to the user's head movements, it can make the experience immersive. The visual output could be in the form of two video feeds, one per eye, presented to the user to form an apparent three-dimensional (3D) view.
The processor generates the visual output using computer-generated imagery (CGI) techniques. One approach to generating CGI output is for a processor to access a memory containing geometric models of objects and a scene description. A scene description might indicate positions in a virtual space for various objects. The objects can be solid objects, light sources, background, texture, etc. as is commonly known in the art of CGI. A renderer, which might be implemented as program code executed by the processor or a separate graphics chip, takes these objects, their locations, and parameters of a virtual “camera” (e.g., a specified viewpoint that is a position in the virtual space where the virtual camera is “located,”, a specified view direction that is the camera angle of the virtual camera, and possibly also bounds of the camera's viewing window). From data representing those elements, the renderer determines (perhaps once every 90th of a second) where everything in the scene is relative to the camera parameters and generates an image (one for each eye in the case of 3D viewing).
For an immersive experience, it is often desired that the coordinate system of the virtual space be fixed relative to a coordinate system of the real-world space being occupied by the user. If the user rotates the user's head, the processor would want to shift the camera angle aligned with the user's head movements, so that the user perceives that the scenery is not moving relative to the user. Where there is the ability for the user to move about in the real-world space (or move on an apparatus that provides for user movement even if the user is actually fixed in one place), the processor would want to shift the position of the viewpoint of the virtual camera accordingly so that the displayed scenery moves consistent with the user's perception of the user's own movement in the real-world space.
For the processor to know how to make those adjustments to the camera viewpoint and view angle, the processor is supplied with information about movements. This might be in the form of sensor data from accelerometers and/or gyroscopes mounted on the headgear and other sensors to detect movement. Thus, a VR rig might comprise a headgear with rotation sensors, other motion/orientation sensors, a processor, program code for generating the necessary imagery, and the like. Some VR HMD might use other techniques for positioning the HMD and other controllers in a room-scale space.
Conventional VR and avatar animation systems have their uses, but improvements are still desirable.